feat(WTinyLFU): adapt the window size with a hill climber by marcft · Pull Request #385 · 1a1a11a/libCacheSim

marcft · 2026-11-08T10:54:31Z

What does this PR do?

Hi, this is the hill climber I proposed in #21. To be short, basically it's great at fixing a bad 1% window and not so great at leaving a good one alone, so I'd keep it opt-in (I put it behind -e hill-climb=1). If it's off, results are the same as develop.

Every 10x cache size of requests it compares the hit ratio with the last period, moves the window 5% (1% to 80%), and turns around if it dropped. It helps a lot when 1% is way too small, like Twitter cluster26 at 0.01 where the miss ratio goes from 0.43 to 0.08. But when 1% is already the best it kind of wanders off and gets worse, like MSR prxy_0 at 0.01 going from 0.22 to 0.29. On Zipf it's a bit worse too, since there's nothing to climb.

I have to say alibabaBlock_117 at 0.01 surprised me, it is worse than every fixed window. So I don't think the issue is picking a bad size, I think it's the moving itself that costs, because every time the window grows main has to throw away 5% of the cache.

Next I want to try a couple things Caffeine does, move objects into the window instead of throwing them away, and make the step smaller over time so it stops wandering. I can also do the indicator version if you want it (the one that sets the window directly from how skewed the workload looks, instead of climbing).

Closes #21

How it works

window-size sets where the climber starts, and main has to be LRU or SLRU.

SLRU_resize() changes SLRU's size and keeps each segment's share. When the window grows, main evicts down to its new size.
SLRU_find() now leaves an object in place if it fits neither its segment nor the next one (only possible after a resize), so a hit can't evict it inside find.
Differences from the paper, also in the file header: the window stops at 1% (at 0% everything misses), a tie keeps the direction like Caffeine (flipping on ties got stuck when the small windows had 0 hits), and the period is in bytes.

How was it tested?

4 unit tests and test_cli.sh cases for bad values. ctest 15/15, test_cli.sh 428/0, the new tests are clean under ASan/UBSan. With it off, WTinyLFU and plain SLRU give the same results as develop (11 fixed-window configs, 32 SLRU runs).

ctest --test-dir _build --output-on-failure passes
Build is warning-free (CI uses -Wall -Wextra -Werror)

Results

26 cache_dataset traces with 0.001, 0.01, 0.1, object size ignored, on develop. Miss ratio change vs the fixed 1% window, in pp (negative is better):

size	better / same / worse	median	mean	worst
0.001	13 / 3 / 8	-0.76	-3.44	+2.15
0.01	14 / 1 / 11	-0.11	-1.24	+6.12
0.1	8 / 5 / 13	+0.06	-0.21	+1.66
all	35 / 9 / 32	-0.00	-1.58	+6.12

Most losses are small: 24 of the 32 worse cells are under 1 pp.
On average it's 1.09 pp behind the best fixed window.
alibabaBlock_117 at 0.01: 0.6382 with the climber, while the fixed windows go from 0.598 (1%) to 0.637 (80%).
Zipf (alpha 0.8 and 1.0, 20M requests, 1M objects): 0.22 to 1.69 pp worse.
Left out: 2 Tencent cells with under 100 objects, where the 1% window rounds to 0 objects.
Throughput (Twitter cluster13 at 0.01): 4.03-4.06 MQPS vs 4.76-4.87 on develop, about the same as a fixed 5-80% window (4.07-4.34), so it's the bigger window and not the climber.

cachesim <trace> oracleGeneral wtinylfu 0.001,0.01,0.1 --ignore-obj-size=1 -e hill-climb=1 (or -e window-size=<w>).

Per-trace miss ratios

trace	size	fixed 1%	climb	best fixed (window)
alibabaBlock_117	0.001	0.7348	0.7367	0.7225 (5%)
alibabaBlock_117	0.01	0.598	0.6382	0.598 (1%)
alibabaBlock_117	0.1	0.4675	0.4841	0.4615 (15%)
alibabaBlock_30	0.001	0.2567	0.2197	0.2097 (15%)
alibabaBlock_30	0.01	0.1804	0.1743	0.1745 (5%)
alibabaBlock_30	0.1	0.1067	0.0823	0.0716 (80%)
w06	0.001	0.4762	0.4977	0.475 (10%)
w06	0.01	0.3243	0.3263	0.3216 (20%)
w06	0.1	0.0894	0.0607	0.0586 (30%)
w12	0.001	0.1243	0.111	0.1059 (30%)
w12	0.01	0.0799	0.076	0.0744 (20%)
w12	0.1	0.022	0.0227	0.022 (1%)
w18	0.001	0.7929	0.8101	0.7905 (5%)
w18	0.01	0.6911	0.6987	0.6905 (5%)
w18	0.1	0.5577	0.5743	0.5577 (1%)
msr_hm_0	0.001	0.5042	0.5022	0.4893 (20%)
msr_hm_0	0.01	0.4068	0.4097	0.4019 (15%)
msr_hm_0	0.1	0.2898	0.2921	0.2892 (5%)
msr_prn_0	0.001	0.3972	0.3706	0.3657 (40%)
msr_prn_0	0.01	0.3113	0.3083	0.3043 (20%)
msr_prn_0	0.1	0.2199	0.2208	0.2173 (5%)
msr_proj_2	0.001	0.9843	0.984	0.982 (40%)
msr_proj_2	0.01	0.9334	0.9372	0.9245 (80%)
msr_proj_2	0.1	0.8945	0.8851	0.8511 (80%)
msr_prxy_0	0.001	0.8328	0.8334	0.8272 (20%)
msr_prxy_0	0.01	0.2242	0.2854	0.2242 (1%)
msr_prxy_0	0.1	0.0435	0.0476	0.0435 (1%)
msr_src1_0	0.001	0.9933	0.9931	0.9885 (80%)
msr_src1_0	0.01	0.9657	0.9667	0.9589 (30%)
msr_src1_0	0.1	0.7769	0.781	0.7769 (1%)
msr_usr_1	0.001	0.8273	0.806	0.7951 (30%)
msr_usr_1	0.01	0.4796	0.4782	0.4731 (10%)
msr_usr_1	0.1	0.3823	0.3864	0.3671 (30%)
meta_reag	0.001	0.4498	0.4267	0.4216 (40%)
meta_reag	0.01	0.3932	0.3856	0.3772 (30%)
meta_reag	0.1	0.3364	0.3437	0.3224 (30%)
meta_rnha	0.001	0.7004	0.6622	0.6551 (40%)
meta_rnha	0.01	0.6077	0.6025	0.5861 (40%)
meta_rnha	0.1	0.4941	0.5021	0.4782 (20%)
meta_rprn	0.001	0.6251	0.5831	0.5764 (40%)
meta_rprn	0.01	0.534	0.531	0.5156 (40%)
meta_rprn	0.1	0.4429	0.443	0.4295 (20%)
meta_kvcache_traces_1	0.001	0.2894	0.2913	0.2814 (20%)
meta_kvcache_traces_1	0.01	0.1626	0.166	0.1563 (15%)
meta_kvcache_traces_1	0.1	0.0817	0.0819	0.078 (20%)
block_traces_1	0.001	0.688	0.527	0.5212 (80%)
block_traces_1	0.01	0.5443	0.5222	0.4769 (80%)
block_traces_1	0.1	0.5102	0.4929	0.461 (80%)
block_traces_3	0.001	0.6919	0.5481	0.5387 (80%)
block_traces_3	0.01	0.5594	0.5304	0.5057 (80%)
block_traces_3	0.1	0.5295	0.5155	0.4893 (80%)
tencentBlock_11775 (left out, 68 objects)	0.001	1.0	0.489	0.4617 (10%)
tencentBlock_11775	0.01	0.0899	0.061	0.0594 (40%)
tencentBlock_11775	0.1	0.0393	0.0379	0.0367 (15%)
tencentBlock_1243 (left out, 61 objects)	0.001	1.0	0.161	0.1472 (5%)
tencentBlock_1243	0.01	0.0251	0.0278	0.0246 (5%)
tencentBlock_1243	0.1	0.004	0.0044	0.0039 (5%)
cluster10	0.001	0.5	0.4999	0.4999 (5%)
cluster10	0.01	0.4999	0.4999	0.4999 (1%)
cluster10	0.1	0.4999	0.4999	0.4999 (1%)
cluster13	0.001	0.6431	0.6284	0.6281 (80%)
cluster13	0.01	0.6289	0.6282	0.6279 (40%)
cluster13	0.1	0.628	0.628	0.6279 (5%)
cluster26	0.001	0.2573	0.1221	0.0922 (80%)
cluster26	0.01	0.4256	0.0843	0.0795 (30%)
cluster26	0.1	0.0824	0.0802	0.0789 (20%)
cluster3	0.001	0.193	0.2004	0.1884 (5%)
cluster3	0.01	0.0791	0.0655	0.0578 (15%)
cluster3	0.1	0.015	0.0162	0.0147 (10%)
cluster45	0.001	0.4889	0.4904	0.4803 (20%)
cluster45	0.01	0.4315	0.4273	0.4233 (40%)
cluster45	0.1	0.3788	0.3798	0.3714 (20%)
cluster50	0.001	0.9376	0.7021	0.6968 (15%)
cluster50	0.01	0.5598	0.5652	0.5578 (5%)
cluster50	0.1	0.2281	0.2001	0.1846 (80%)
wiki_2019t	0.001	0.7116	0.7271	0.7098 (10%)
wiki_2019t	0.01	0.4442	0.4616	0.4405 (5%)
wiki_2019t	0.1	0.1832	0.1876	0.1832 (1%)
Zipf alpha 0.8	0.001	0.8088	0.8257	0.8088 (1%)
Zipf alpha 0.8	0.01	0.6624	0.6736	0.6624 (1%)
Zipf alpha 0.8	0.1	0.4293	0.4375	0.426 (5%)
Zipf alpha 1.0	0.001	0.504	0.5201	0.5038 (5%)
Zipf alpha 1.0	0.01	0.3439	0.3471	0.3439 (1%)
Zipf alpha 1.0	0.1	0.1867	0.1889	0.1867 (1%)

Type of change

Other: new option for WTinyLFU

Checklist

Formatted with clang-format (or the pre-commit hook from scripts/setup_hooks.sh)
Added or updated tests
Added or updated documentation

Add hill-climb=1, the window hill climber of Adaptive Software Cache Management (Middleware'18). Every 10 x cache size bytes of requests it compares the hit ratio with the previous period and moves the window 5% (between 1% and 80%), turning around when the hit ratio dropped. The window already holds whatever the main cache does not use, so a resize only sets the main cache's size, and the main cache evicts down to it when the window grows. SLRU_resize() keeps each segment's configured share, and SLRU_find() keeps an object that no longer fits the next segment in place instead of promoting and evicting it. Off by default, and results without hill-climb=1 are unchanged. Only LRU and SLRU main caches are supported.

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

SLRU_find() now keeps an object in place only when it fits neither its segment nor the next one, which can only happen after SLRU_resize(). Before, plain SLRU with a smaller later segment (seg-size=9:1) changed slightly. WTinyLFU's 1:4 main behaves the same as before. Also say in the header that growing the window evicts from the main cache, and list hill-climb=0 in the default params.

marcft · 2026-11-08T14:37:08Z

Just tried making the climber step smaller over time, and also the paper's indicator (setting the window directly from how skewed the workload looks, so no climbing). Surprisingly the step barely matters, and the indicator is actually worse, so I would just keep the PR like this.

About alibabaBlock_117, my guess was wrong. I tried moving objects into the window instead of throwing them away and it got worse (0.638 to 0.649). The window also stays between 1% and 35% the whole time, so it's not picking a bad size either. So I still don't know what's causing the extra misses there 😅

Numbers (same 26 traces)

	mean vs 1% (pp)	gap to best fixed (pp)
this PR	-1.58	1.09
smaller step over time	-1.60	1.08
move instead of evict	-1.56	1.12
both (Caffeine's rule)	-1.60	1.07
paper's indicator	-0.99	1.69

The smaller step and the indicator help on Zipf (0.96 pp worse than 1% goes to 0.46 and 0.19), but the indicator gets cluster26 at 0.001 badly wrong (0.12 to 0.42). The indicator is ported from Caffeine's simulator.

1a1a11a · 2026-11-08T15:00:50Z

Awesome work!

marcft requested a review from haochengxia as a code owner November 8, 2026 10:54

Copilot AI balanced review requested due to automatic review settings November 8, 2026 10:54

Copilot AI reviewed Nov 8, 2026

1a1a11a merged commit b3ea4f0 into 1a1a11a:develop Nov 8, 2026
7 checks passed

Navigation Menu

Repository navigation

feat(WTinyLFU): adapt the window size with a hill climber - #385

feat(WTinyLFU): adapt the window size with a hill climber#385
1a1a11a merged 2 commits into
1a1a11a:developfrom
marcft:feat/wtinylfu-hill-climb

marcft commented Nov 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

marcft commented Nov 8, 2026

Uh oh!

1a1a11a commented Nov 8, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

marcft commented Nov 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

How it works

How was it tested?

Results

Type of change

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

marcft commented Nov 8, 2026

Uh oh!

1a1a11a commented Nov 8, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

marcft commented Nov 8, 2026 •

edited

Loading